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Abstract 

Reusable learning objects are an approach that is receiving a significant amount of attention in 
distance-based and online education (see Reports #11, 40, and 46 in this series). They have the 
potential to provide cost-effective, personalised instruction with a short development time. 
Instructional design principles, however, must play an important part in any such development 
effort, within a design process that occurs on two levels. At the higher level, instruction must be 
designed to deliver material efficiently to students at the modular/ course/ programme level. 
Design principles should be applied at the secondary level, at which the unique characteristics of 
learning objects are determined. Various instructional design (ID) methodologies are capable of 
dealing with these issues. The current report discusses a sle of these methodologies, and compares 
the ID adequacy of objects in four major learning object repositories: Merlot, CLOE, EOE, and 
Wisconsin Online. At the time of writing, each of these repositories contains objects that are 
inadequate from the ID point of view. 

Instructional Design Requirements of Learning Objects 

For a learning object (LO) to have instructional impact, it must embody explicit planning for 
learning, intentional instructional design (ID). Solid ID is a critical part of reusable LO design 
(Longmire, 2000; Wiley, 2000; Douglas, 2001; and Sosteric and Hesemeier, 2002). For the 
purposes of the current review, it will be assumed that the term LO refers to a digital entity 
intended to further the achievement of a specific learning objective. This working definition 
discounts those LOs that generate learning serendipitously, and could restrict the review to LOs 
in computer-based environments. Digital entities, whose primary purpose in a given context are 
to provide information, will be referred to as content objects (CO). Depending upon the context, a 
CO may become a LO or may serve as a LO component. 

LOs typically comprise two different major components that may, or may not be, co-resident on 
the same computer - the learning content and the metadata. Both of these LO aspects must be 
considered during the ID process for the object to be effective. The metadata provide the learning 
context for the LO, and are the key to its reusability. The prime requirement of a LO is that it is 
reusable in different contexts (Sicilia and Garcia, 2003), and specifically in each of its target 
contexts as defined in the metadata. The granularity of an LO is defined as its instructional size, a 
characteristic hotly debated among LO advocates (Wiley, 2000). As learning objectives may be 
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needed at different levels of a course or programme, each combining different lower-level 
enabling objectives, so too a LO can be defined as material capable of assisting a higher-level 
learning objective, comprising lower-level LOs defined to achieve enabling objectives. From the 
ID perspective, it is the scope of the learning objective that is important in the definition of LO 
granularity. The smaller the granularity, and the finer the objective the LO is designed to achieve, 
the greater the LO’s reusability, for the object will be applicable across a greater number of 
learning contexts and adaptable to different learner characteristics. Lesser granularity permits the 
LO’s use in an environment where LOs can be moved in and out of the learning system in 
response to changes in learner type. Reduced granularity, however, increases cost and 
management difficulty (Herridge Group, 2002). 

For a LO to be reusable across multiple learning contexts, it must also be able to stand on its own. 
For the LO to serve different objectives in different instructional contexts, it must be independent 
of the position in which it is placed in the larger learning environment. If a LO is dependent upon 
other objects that appear before or after it in the learning sequence, it cannot achieve a learning 
objective in its own right. It is therefore not an LO at all, but is - at best - an asset in the structure 
of a larger LO, perhaps involving other LOs that are necessary to the achievement of the learning 
objective. The LO’s ability to be used in different technological environments enhances its 
reusability; efforts must be made during the LO’s development to maintain its independence of 
any particular hardware or software environment, including the use of specialised plug-ins. Such 
technological dependencies should be recorded in the LO’s metadata. To accomplish all of these 
objectives, it may be necessary to adjust the ID methodology used. 

Instructional Design Overview 

Instructional design (ID) is “the systematic and reflective process of translating principles of 
learning and instruction into plans for instructional materials, activities, information resources, 
and evaluation” (Smith and Ragan, 1999). Instruction is normally regarded as a system of 
interdependent elements that facilitate intentional learning (Gagne, Wager, Golas, and Keller, 
2005; Reiser and Dempsey, 2002). ID methodologies can be applied at many levels, from the 
development of entire curricula, to that of the activities for a single lesson. It can represent the 
work of an individual or of a team of specialists. ID models are usually represented as a sequence 
of iterative processes, often requiring a number of cycles, before the product is fully refined. 
Depending upon the complexity of the instruction, however, ID may be better represented as a 
tightly interwoven knot than as a linear sequence of processes. This interwoven nature is 
particularly evident in those designs produced by rapid prototyping techniques (Smith and Ragan, 
1999). In such cases, care must be taken to ensure the adequate completion of each stage before 
the start of the next; otherwise critical information may be missing from the material, resulting in 
learning errors. 

The generic form of ID, found at the core of most current models, is the ADDIE model - 
Analysis, Design, Development, Implementation, and Evaluation (Gagne, Wager, Golas, and 
Keller, 2005; Reiser and Dempsey, 2002). This model focuses on learner performance relating to 
real-world tasks, and makes no assumption that a live teacher is necessary (Reiser and Dempsey, 
2002). These characteristics make the ADDIE model appropriate as a starting point for the 
development of online learning and, specifically, LOs. Other methodologies have been proposed 
for the development and use of LOs, though have not differed essentially from the basic ID model 
(Martinez, 2000; Herridge Group, 2002; IMS, 2003; Downes, 2003; Johnson, 2003). LO design 
and development has two stages: the development of the LOs themselves, and their organisation 
into large aggregations, whether larger objects, modules, or courses. Most writers deal with the 
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aggregation of existing LOs rather than with the development of original LOs. ID principles can 
apply to either level. Johnson (2003) suggests that LOs should he developed according to 
approaches that ensure the appropriate application of sound ID strategies, enforcing clearly 
defined quality standards. A clear learning intent must he associated with a LO, and such intent 
cannot he made consistently evident in the absence of a comprehensive and systematic ID 
process. Martinez (2000) suggests that ID principles that result in better instruction have been 
avoided by LO designers, largely because of lack of familiarity with them. She questions the 
instructional value of LOs developed without due attention to major ID concepts. As metadata 
standards evolve, it is to be hoped they will become capable of including more ID information. 

A Comparison of LO Requirements 

The design considerations of a LO first arise at the initial analysis stage, at which the 
characteristics of the learners are determined, and the contexts in which they will learn. Analysis 
must consider the students’ ability to learn effectively using online education, and the extent to 
which they will be able to endure visual and design differences among LOs brought together from 
disparate sources. Learner characteristics should be clearly indicated in the LO’s metadata, since 
this will affect the appropriate reuse of the LO. In designing a monolithic course or module, 
learning objectives are defined in terms of “the skills to be learned, the conditions under which 
the skills must be performed, and the criteria for successful performance” (Dick, Carey, and 
Carey, 2001). To make these objectives appropriate for the development of LOs, they must be 
broken down into more detailed enabling objectives facilitating the development of information 
LOs and lower-level LOs. During the development of the instructional strategy, the specific 
design of the LO must be addressed. The manner in which learners will be presented with the 
instruction is determined (the delivery media and learning activities), and the sequencing and 
aggregation of objectives. These tasks must be completed whether the designer is developing an 
entire course or individual LOs. The size of each LO must be decided and its reusability weighed 
against the applicability of the LO in the instructional context. Smaller LOs can be aggregated 
into larger granularity LOs, though this may involve greater management costs. Wiley (2000) and 
Dick, Carey, and Carey (2001) have each recommended consideration of Gagne’s nine events of 
instruction during the development of an instructional strategy. 

A further consideration when developing an instructional strategy appropriate to the use of LOs is 
that multiple approaches to the instruction may be necessary to account for an LO’s use in 
multiple contexts with multiple learning styles. For exle, an LO may include video, audio, and 
textual treatments of certain material to assist those students who learn best from one or another 
of these media. During the development of instructional materials for traditional classroom 
delivery, both student and instructor materials may be developed. Student materials will contain a 
summary of the knowledge to be acquired, and instructor materials will provide elaborative 
information. In developing online learning materials, however, such a division of information 
may not be appropriate. In the development of LOs specifically, the developer may have no 
knowledge of their ultimate usage, and must therefore make the information contained within the 
object as complete and context-free as possible, without sacrificing effectiveness. Such broad 
applicability implies a strong organisation of information, and the likely use of multiple media to 
ensure that the LO is independent of external requirements. 

In order to enhance its transferability to multiple contexts, development of an effective LO also 
involves serious consideration of human-computer interface issues (Cassarino, 2003). Formative 
and summative evaluation of LOs and their aggregation can be carried out as in traditional 
teaching contexts. While traditional course materials can be evaluated in the context for which 
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they were originally designed, LOs need to be evaluated in multiple contexts. Since the original 
designer cannot anticipate all of the contexts in which an LO will be used, evaluative data may 
need to be obtained from designers who have included the LO in other learning situations. 
General metadata standards do not include fields for feedback of this type, though some LO 
repositories (e.g., Merlot, see below) have added this feature to their structure. 

LO Repositories and ID Principles 

Though it is often difficult to determine if ID standards have been deliberately applied to a 
particular piece of instructional material, objects contained in content repositories can be assessed 
in terms of their adherence to specific ID principles. In particular, the following questions can be 
asked of the individual items in an object repository: Do they facilitate intentional achievement of 
specific instructional objectives? Or are they merely COs, supplying basic information only and 
needing to be combined with other objects to facilitate intentional learning? The object’s ease of 
use, navigability, and reusability can be assessed by reviewing its interface and the extent to 
which it is tied to a particular learning context. 

1. Merlot Multimedia Educational Resource for Learning and Online 
Teaching 

Merlot Multimedia Educational Resource for Learning and Online Teaching 
( http://www. merlot.org) is a major international repository administered by a consortium of 
higher education institutions, and contains objects on a broad spectrum of disciplines. Objects 
added to the repository are open for peer review, which are made available to visitors along with 
the metadata that assist in assessing the applicability of an object for specific purposes. The 
objects are not actually contained within the repository, and it is often difficult to contact their 
source sites. This reviewer surveyed the first 20 objects presented in Merlot’ s Business category, 
listed with reference to peer ratings. Of these, nine were content objects (COs) that would need to 
be combined with other instructionally specific objects in order to supply the learners with both 
context and intent. Only eight objects could be classified as actual LOs, four of which appeared to 
be widely reusable, though only as whole topics owing to their granularity. Many of the object 
sites were commercial, containing information provided primarily as a means to sell another 
product or service. A scan of objects listed further down the list in the Business category revealed 
many other commercial sites presenting COs with apparent limited reusability. 

2. CLOE Cooperative Learning Object Exchange 

CLOE Cooperative Learning Object Exchange (http://cloe. on. ca) is a repository at the University 
of Waterloo containing peer reviewed LOs. All objects in the repository undergo peer review 
before being posted. Their metadata are available, but the results of the peer reviews are not. This 
restricts the formative evaluation of an object by other than its original creators. CLOE is a small 
repository, so the current object review was cross-disciplinary. Of the 12 objects reviewed, seven 
were classifiable as COs, requiring other information in order to produce intended learning. Four 
were full-topic, stand alone LOs with definitions of objectives, content presentation, and 
assessment. One object was unavailable, an unexpected problem given that the objects are all 
housed within the repository database. None of the objects were simple graphic assets as with the 
image files found in some repositories. All were self-contained and would be highly reusable in 
appropriate contexts. Their metadata were not always complete, and a potential user would need 
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to review each object in turn to find adequate comparative information. Overall, the quality of 
materials in this repository appears to be higher than that found in Merlot. 


3. EOE Electronic Object Economy 

EOE Electronic Object Economy (http://www. eoe. ors) is a LO repository containing Java-based 
objects. Objects in other repositories may be developed using this programming environment, and 
the repository is distinctive in that, except for the HTML shells surrounding the Java applets, no 
other technologies are used. The Business category of objects was again chosen for review, and 
eight objects were selected for assessment. Of these, two were inelegant, but functional and 
reusable objects incorporating simple Java applets for performing calculations. While attempting 
to navigate to one object that claimed to calculate amortisation schedules, the evaluator was 
redirected to a pornography site. The remaining sites were unavailable. The review was 
terminated at this point since the search was totally unproductive. 

4. WORC Wisconsin Online Resource Center 

WORC Wisconsin Online Resource Center ( http://www. wise-online. com) is a collection of Flash- 
based COs. Each is short and only occasionally provides complete topic coverage. Six objects 
were reviewed in the Business category. Each displayed a consistent interface that would assist 
their seamless integration into a comprehensive course. Eor most purposes, however, they are 
unlikely to stand on their own. 


Conclusions 

Reusable EOs can be effective in the delivery of cost-effective, timely, reusable instructional 
materials. Their effectiveness, however, is as much a product of efficient instructional design as 
any learning materials or process; and ID principles should addressed during the objects’ design 
and development. The ID principles proposed for use in object-oriented situations are essentially 
the same as those used in the development of traditional instructional materials. They should be 
applied not only in the design and development of modules and courses using EOs, but also in the 
creation of the objects themselves. 

A survey of objects contained in existing repositories, however, reveals that relatively few can 
strictly be defined as EOs at all, being of a basic CO type and not useable on a stand-alone basis 
to bring about intentional learning. These objects need to be aggregated with other materials in 
order to achieve specific objectives. Their quality also varies greatly, and in some cases calls into 
question the professionalism associated with the learning materials of which they are a part. The 
current review indicates that the effective development of EOs requires the clear definition of an 
instructional process addressing the unique characteristics of EO technologies, within the 
structured process stressed by ID principles. If such principles are not heeded, learning 
repositories will gain a reputation for amateurish content, rather than credibility as worthwhile 
educational resources. 
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The next report in the series discusses techniques for optimizing conferencing freeware. 


Technical Evaluation Report 49 
Learning Objects and Instructional Design 


7 


N.B. Owing to the speed with which Weh addresses are changed, the online 
references cited in this report may he outdated. They can he checked at the 
Athabasca University software evaluation site: http://cde.athahascau.ca/softeval/ . 
Italicised product names in this report can he assumed to he registered 
trademarks. 
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